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Charles  Hitch 


Military  systems  analysis  is  an  extension  of  operations 
research  techniques  of  World  War  II  to  problems  of  broader  context 
and  longer  range--e.c.,  force  composition  and  development  as  well 
as  operations  decisions. 

Greater  complexity  is  inevitable  as  we  a'  tempt  analyses  to 
aid  decisions  affecting  more  distant  time  periods:  growth  In 
number  of  relevant  variables;  compounding  of  uncertainties; 
increased  importance  of  enemy  reactions;  complications  of  time 
phasing;  the  need  for  a  broader  concept  of  criteria. 

Techniques  available  to  the  systems  analyst  for  dealing  with 
these  complexities  are  less  than  satisfactory.  "Factoring"  is 
Inevitable.  All  routine  or  mechanistic  approaches  are  deficient; 
e.g.,  expected  value  and  minimax  criteria.  Design  and  criteria 
problems  are  of  dominant  importance. 

The  systems  analyst  may  have  to  be  content  with  better  rather 
than  optimal  solutions;  or  with  devising  and  costing  sensible 
methods  of  hedging;  or  merely  with  discovering  critical  sensitivi¬ 
ties.  He  has  an  Important  role  as  inventor  of  systems. 

The  difficulties  of  systems  analysis  are  rooted  in  the  nature 
of  the  military  problems.  Other  methods--e.g. ,  piecemeal  analysis; 
intuition--do  not  escape  them  and  have  limitations  of  their  own. 
Military  systems  analysis  provides  a  framework  for  combining  the 
knowledge  >^f  experts  in  many  fields  to  reach  solutions  which 
transcend  any  individual  expert’s  Judgment. 


The  first  widespread  and  explicit  use  of  scientific  method  or  analysis  as 
an  aid  to  military  decision  making  was  made  by  operations  analysis  (or  operations 
research)  teams  in  World  War  II.  Small-scale  and  partial  use  of  similar  analytic 
techniques  had  long  been  common.  Thucydides  describes  examples  of  their  use  by 
the  Athenians  in  the  Peloponnesian  wars. 

♦This  paper  la  a. condensation  lectures  prepared  for  Air  Foice  audiences; 
.-drt~~ga:B /presented -^^the  Operations  Research  Society  of  America  a3ES6tee  Los 
Angeled meeting 
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But  even  World  War  II  operations  analyses  were  limited  in  character. 

They  related  to  operations  in  the  Immediate  future,  not  to  force  composition 
or  the  development  of  equipment  which  would  affect  operations  in  the  more 
distant  future.  Partly  in  consequence,  they  were  simple,  in  the  sense  that 
they  considered  only  a  emaxl  number  of  Interdependent  factors.  They  were  able, 
as  a  rule,  to  use  some  proximate,  obvious  stral^ntforward  objective  or  criterion 
as  a  basis  for  chooslnej  one  operation  over  another. 

A  typical  example  of  a  World.  War  II  operations  analysis  problem  was  wnat 
bomber  formation  to  use  in  attacking  targets  deep  in  Germany.  This  problem 
had  few  variables  and  an  obvious  criterion:  minimize  losses  in  achieving  given 
target  destruction. 

Since  the  war,  at  RAND  and  elsewhere,  attempts  have  been  made  to  use 
analysis  as  an  aid  to  military  decisions  in  problems  of  immensely  t,reater 
difficulty  and  complexity.  There  has  been  a  tendency  to  use  the  term  "systems 
analysis"  to  describe  these  more  complex  analyses,  but  tnere  is  no  line  of 
demarcation.  Both  operations  aI^aly8i3  and  systems  analysis  are  attempts  to 
apply  scientific  method  to  Important  proble.ms  of  military  decision,  even 
though  the  probiems  are  not  particularly  appropriate  for  scientific  method  and 
would  never  be  selected  for  the  application  of  scientific  method  by  a  truly 
"academic"  researcher. 

Both  operations  analysis  and  systems  analysis  have  the  same  essential 
elements : 

An  objective  or  objectives  which  we  desire  to  accomplish. 

Alternative  techniques  or  Instrumentalities  (or  "systems" ) 
by  which  the  objective  may  be  accomplished. 

The  "costs"  or  resources  required  by  eticn  system. 
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A  mathematical  model  or  models;  l.e.,  the  mathematical  or 
logical  framework  or  set  of  equations  showing  the 
interdependence  of  the  objectives,  the  techniques  and 
instrumentalities,  the  environment,  and  the  resources. 

A  criterion,  relating  objectives  and  costs  or  resources,  for 
choosing  the  preferred  or  optimal  alternative. 

THE  DEVELOPMENT  OF  MILITARY  SYSTBiS  ANALYSIS 

Development!!;  and  extensions  of  operations  analysis  since  World  War  II  have 
taken  the  following  forma: 

1. -  The  use  of  analysis  to  aid  In  force  composition  and  development  as 

well  as  operations  decisions. 

2.  A  great  increase  in  number  of  interdependent  factors  considered. 

5.  The  explicit  treatment  of  problems  of  uncertainty. 

Tae  explicit  treatment  of  enemy  reactions. 

5.  The  explicit  treatment  of  time-phasing. 

6.  A  broader  concept  of  objectives  and  criteria  appropriate  to  the 
broader  and  longer  range  problems  of  decision  being  analyzed. 

We  will  discuss  these  in  turn,  considering  in  an  impressionistic  and  superficial 
way  the  analytic  techniques  apprv-priate  for  each  extension. 

1.  Force  Composition  and  Development  Decisions 

Models  used  for  force  composition  and  development  decisions  need  not  be 
formally  different  fr-^m  those  used  for  operations  decisions.  Their  form  Is 
likely  to  differ,  however,  because  of  the  greater  number  of  interdependent 
variables,  the  increase  In  uncertainty,  and  the  broader  criteria  appropriate 
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to  the  l.inger  ti;oe  horizon  considered.  As  an  historical  fact,  the  attempt  to 
apply  models  to  development  problems  has  sparked  Important  developments  in 
systems  analysis  methods  in  all  these  areas. 

2.  More  Variables 

The  application  to  force  composition  ajid  development  decisions  means  that 
we  are  concerned  with  a  military  force  two  or  three  to  ten  or  fifteen  years  in 
the  future  instead  of  with  present  forces.  This  fact  alone  vastly  increases 
the  number  of  interdependent  variables  which  we  have  to  consider.  All  sorts  of 
thin^jS  which  are  given  in  the  short  run  become  variables  in  the  long  run.  For 
example,  in  the  bomber  formation  operations  analysis  in  World  War  II,  the  planes 
were  B-17*e,  their  number  was  given,  the  targets  were  given,  the  bombs  were 
given,  the  enemy  defenses  were  given,  etc.  In  the  longer  run  these  are  not 
given.  They  are  unknown.  They  become  variables.  Some  are  variables  subject 
to  our  control,  some  are  variables  subject  to  the  enemy’s  control,  some  are 
subject  to  nobody's  control.  But  all  are  variables.  Moreover  they  are  all 
interdependent . 

To  illustrate  how  this  can  increase  the  number  of  interdependent  variables 
and  systems  to  be  compared,  consider  a  simple  bomber  or  missile  development 
problem.  Suppose  we  ruthlessly  simplify  aircraft  characteristics  to  three-- 
speed,  range,  and  altitude.  Wixat  else  do  we  have  to  consider  as  interdependent 
variables?  At  least  bomber  formation,  flight  patn,  base  system  (two  varlables-- 
number  and  general  location),  target  system,  bombs,  enemy  defenses  (two 
variables?).  This  may  not  sound  like  many  (in  fact,  it  is  far  fewer  tnun  are 
probably  necessary),  but  if  we  go  no  higher  than  ten,  and  if  we  let  each 
variable  take  only  two  alternative  values,  we  already  have  2^*^ 
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oases  or  systems 
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to  calculate  and  coaipare  (2^^  >  1,000).  If  we  let  each  variable  take  four 
alternative  values,  we  have  cases  (4^^  >  1,000,000). 

In  the  World  War  II  bomber  formation  problem,  on  the  same  assamptiona, 
everything  else  is  given  except  possibly  flight  path.  The  number  of  cases, 
with  the  same  degree  of  simplification,  would  be  two,  four,  or  at  most  sixteen. 
So  Just  moving  into  the  future  puts  us  in  a  different  ball  park. 

What  do  we  do  when  we  are  cjn fronted  with  a  million  or  billion  or 
decllllon  cases  to  compute  and  compare V  Of  course  we  develop  higher  speed 
computers  every  year  and  greater  skill  in  using  them,  and  I  do  not  want  to 
belittle  the  significance  of  this  accomplishment.  But  even  the  capacities 
of  modern  high-speed  computers  are  limited.  We  still  have  to  cut  down  moat 
of  our  problems  for  this  reason.  And  there  are  usually  more  confining 
limitations.  We  have  to  get  relevant  data  and  relations  to  feed  into  the 
machine  on  all  these  systems.  The  more  variables  we  deal  with,  the  more  time 
this  takes  and  the  more  pecple--both  scarce  and  valuable  resources. 

So  somehow,  for  one  practical  reason  or  another,  big,  broad  problems  have 
to  be  cut  down  in  size.  We  have  to  "factor  out"  a  practical  problera--i.e. , 
factor  out  those  variables  which  are  especially  important  for  the  decision  with 
which  we  are  concerned,  amd  "suppress"  or  "aggregate"  the  rest.  How  do  we  do 
this?  To  a  limited  extent  by  preliminary  analyses  and  tests,  but  for  the  most 
part  by  sheer  judgment.  It  is  hard  to  do;  it  amounts  to  no  less  than  deciding, 
in  designing  the  analysis,  what  la  Important  and  what  isn't. 

Probably  moat  systems  analyses  that  go  wrong,  go  wrong  here.  Either  they 
include  a  mass  of  data  and  calculations  which  are  just  excess  baggage,  or  they 
exclude  some  really  critical  factor  on  which  a  good  decision  depends. 


1 


P-699 

8-18-55 

-6- 


3.  Explicit  Treatment  of  Uncertainty 

Uncertainty  ie  present  In  operations  problems,  and  It  needs  to  be  dealt 
vlth  there.  But  uncertainties  multiply,  as  an  alasot  invariable  rule,  as  ve 
look  further  In  future. 

We  need  to  distinguish  several  kinds  of  uncertainty: 

(a)  "Planning  factor"  uncertainty.  Planning  factorB--attrltlon 
rates;  average  bombing  errors,  etc. --are  always  uncertain,  but 

less  so  In  the  present  than  In  the  future. 

(b)  Uncertainty  regarding  the  enemy  and  his  reactions.  Like  (a), 
this  1b  always  with  us  and  increases  with  time. 

(c)  Strategic  uncertainty.  Will  it  be  war,  cold  war,  or  peace? 

If  war,  when?  General  or  local?  With  what  political  constraints? 
To  achieve  what  political  objectives?  Who  will  be  our  enemies,  who  our 
allies?  Will  our  allies  make  bases  available  to  us? 

In  the  very  short  run,  strategic  uncertainties  of  this  kind  are 
sometimes  trlvlal--e.g. ,  In  the  typical  operations  analysis  problem 
in  World  War  II.  In  the  longer  run  they  may  quickly  become  dominant. 

Ho  force  composition  or  development  analysis  can  ignore  them.  For 
example,  the  whole  composition  of  the  Air  Force  may  be  drastically 
affected  by  our  opinion  about  the  relative  likelihood  of  big  and  little 
wars.  Who  can  tell  us?  The  answer  Is  not  easy  or  obvious.  Ho  present 
Administration  or  Joint  Chiefs  of  Staff  can  make  strategic  decisions 
binding  on  a  future  Administration  or  Joint  Chiefs  of  Staff. 

(d)  Technological  uncertainty.  This  Is  small  in  the  present;  in  the 
future  it  can  be  vast.  For  example,  until  recently  there  was 


real  uncertainty  as  to  whether  the  H-bomb  would  work,  and  if  so,  when; 
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thl«  profoundly  influenced  the  structure  of  many  systems  analyses. 

There  is  al%raiy8  technological  uncertainty  of  some  degree  attached 
to  research  and  development;  otherwise  it  wouldn't  be  research  and 
development. 

(e)  Finally,  there  is  statistical  \ancertalnty--the  kind  that  would 

still  exist  even  if  wo  could  predict  technological  progress,  the 
dates  of  wars,  enemy  strategies,  and  the  central  values  of  all  Important 
parameters.  Statistical  uncertainty  stems  from  chance  elements  in  the 
real  world.  Statistical  uncertainty  is  the  least  of  our  worries  in 
systems  analysis:  not  only  have  we  made  great  progress  In  Monte  Carlo 
techniques  for  dealing  with  statistical  fluctuations  when  we  have 
to,  but  these  are  usually  swamped  by  uncertainties  regarding  central 
values  and  states  of  the  world  in  long-range  problems.  The  use  of 
elaborate  zoethods  to  reflect  statistical  uncertainties  in  such  problems 
is  likely  to  be  an  expensive  frill. 

What  do  we  do  in  a  systems  analysis  to  take  account  of  the  proliferation  of 
uncertainties  resulting  from  our  ignorance  of  the  future?  We  can't  ignore  them. 
To  base  our  decision  on  some  single  set  of  "best  guesses"  could  be  disastrous. 
For  example,  suppose  that  there  is  uncertainty  about  ten  factors  (e.g.,  will 
overseas  bases  be  available?  will  the  enemy  have  interceptors  effectlw  at 
60,000  feet?),  and  we  make  a  best  guess  on  all  ten.  If  the  probability  that 
each  best  guess  is  right  is  sixty  per  cent,  the  probability  that  all  ten  are 
right  is  one-half  of  one  per  cent.  We  would  be  ignoring  futures  with  a  99»5 
per  cent  probability  of  occurring. 
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The  problem  breaks  down  Into  two  parts: 

(a)  How  compute  all  the  "interesting"  contingencies 7  This 
raises  problems  very  similar  to  those  we  have  already 

discussed.  Mere  computation  time  is  important,  but  usually 
much  more  so  Is  deciding  which  cases  deserve  to  be  computed 
and  assembling  the  data  and  structure  for  each. 

(b)  Once  we  have  computed  the  alternatives,  we  will  almost 
Inevitably  discover  that  one  strategy  is  superior  in  some 

contingencies,  another  in  others.  How  does  the  systems  analyst 
choose  the  preferred  strategy?  What  decision  does  he  recommendV 
For  example,  suppose  analysis  shows  that  a  strategic  bombing  cystem 
dependent  on  overseas  bases  will  be  most  effective  for  I96O-I965.  Suppose 
further  that  we  regard  it  as  quite  likely,  but  not  certain,  that  we  will  have 
overseas  bases  in  that  time  period.  Suppose  finally  that  if  we  should  not  have 
the  bases,  the  system  would  be  very  bad.  What  do  we  do  in  such  a  case? 

Ingenious  efforts  have  been  made  to  find  an  answer,  but  with  limited  success. 
The  shortcomings  of  merely  maximizing  the  expected  outcome  are  too  well  known 
to  elaborate  upon:  because  it  incorrectly  weights  high  and  low  outcomes,  it 
can  lead  to  the  choice  of  a  reckless  strategy  and  possible  catastrophe. 
Maximizing  the  expected  value  or  "utility"  of  the  possible  outcomes  is  not 
subject  to  the  same  criticism  of  recklessness,  but  it  is  rarely  possible  to 
assign  objective  utilities  to  the  outcomes,  even  if  one  somehow  surmounts  the 
theoretical  and  practical  obstacles  of  assigning  probabilities. 

Expected  outcomes  or  values,  in  any  event,  are  appropriate  only  in  playing 
against  nature,  not  In  playing  against  an  Intelligent  opponent;  and  in  few 
military  systems  analyses  is  it  legitimate  to  Ignore  completely  the  possibility 
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of  intelligent  opposition.  Gaioe  theory  suggests  that  in  such  circumstances 
we  should  max -min;  l.e.,  choose  the  system  which  miniralzeB  the  worst  that  can 
happen  to  us.  ^Is  would  be  good  enough  if  we  were  really  playing  a  two-person 
constant  sum  game,  but  we  never  are— except  perhaps  at  the  lowest  operational 
level.  And  if  o\ir  opponent  le  anything  lees  than  completely  wise  and  resourceful 
(l.e.,  always)  max-mining  is  too  conservative;  it  forfeits  opportunities  to 
exploit  enemy  ml stakes. 

Some  advocates  of  max-mtn  have  a.rgued  that  a  special  variety  of  it  — 
max-mining  "regret"  or  "loss"  rather  than  outcojiQ3--i3  an  appropriate  criterion 
or  approximately  rational  rule  of  thumb  for  the  case  when  we  play  against  nature 
and  can  in  prlnlciple  calculate  probabllitieB  but  in  actual  practice  have  only 
vague  estimates.  It  at  least  does  not  lead  to  foolishly  conservative  strategies, 
as  ordinary  max-mining  does  in  these  circumstances.  But,  as  we  have  seen,  we 
never  or  rarely  find  systems  problems  in  which  the  game  element  Is  completely 
absent . 

There  simply  is  no  satisfactory  general  answer  to  the  problem.  Different 
people  take  different  views  of  risks  —  In  their  O'wn  lives  and  as  decision-makers 
for  the  nation.  Some  play  boldly,  some  play  for  safety.  So  what  does  the  poor 
systems  analyst  do(  He  frequently  calculates  some  expected  outcome  or  mlnimaxes 
or  both,  but  in  interpreting  his  results  he  is  aware  of  their  biases  and 
inadequacies.  But  he  doesn’t  stop  there.  There  are  other  tricks  of  the  systems 
analysis  trade  where  uncertainties  are  grossly  important: 

(a)  He  tries  to  invent  a  system  which  is  as  good  or  almost  as 

good  if  overseas  bases  ere  available,  but  still  pretty  good 
if  -they  aren’t.  We  call  a  system  which  is  best  in  any  circumstances 
a  "sure  thing"  or  "dominant." 


We  can  seldom  find  a  truly  dominant 
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system,  but  sometimes  ve  can  come  close.  Such  a  system  in  the 
above  example  might  be  a  bomber  force  wnich  would  normally  use 
overseas  bases,  but  If  they  were  not  available  would  have  a 
substantial  capability  with  air  refueling  from  Z1  bases.  It  is 
arguable  that  the  most  valuable  function  of  systems  analysis  is 
the  stimulation  of  systems  Invention. 

(b)  If  he  fails  to  find  a  dominant  solution,  he  calculates  the  cost 
of  providing  Insurance  against  the  chance  of  cataetrophe--e.g. , 

by  buying  a  mixed  force  with  a  substantial  number  of  very  long-range 
aircraft.  Then  the  Air  Force  has  to  make  a  command  decision,  but  at 
least  it  can  do  so  knowing  what  the  insurance  costs. 

(c)  If  he  la  concerned  with  development  decisions,  he  recommends 
the  development  of  aircraft  and  mlselles  only  some  of  which 

depend  on  overseas  bases.  The  situation  may  be  clearer  when  decisions 
about  quantity  procurement  have  to  be  made  several  years  hence.  We 
can  afford  to  develop  more  types  of  equipment  than  we  procure  in 
quantity,  and  given  the  relatively  low  cost  of  development,  we  ought 
to.  Insurance  is  cheaper  at  this  stage.  The  systems  analyst  must 
guard  against  the  Implicit  "either/or."  Mixed  systems  frequently  are 
right  for  procurement,  even  more  frequently  right  for  development. 

4.  Kxplicit  Treatment  of  Enemy  Reactions 

In  some  problems,  wnat  the  enemy  does  is  obviously  crucial  in  making  the 
right  decision,  e.g.,  the  need  for  SCM,  or  the  best  allocation  of  funds  between 
offense  and  defense.  But  it  can  also  be  crucial  in  majoy  less  obvious  Instances. 
For  example,  the  choice  between  offensive  bombers  and  missiles  is  extremely 
sensitive  to  their  vulnerability  to  enemy  attack--ln  the  air  and  especially  on 
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the  ground.  So  there  la  great  interest  in  developing  models  appropriate  for 
problems  of  this  kind. 

Two  kinds  of  models  are  available: 

(a)  Gsae  theory  models. 

(b)  Games  (i.e.,  war  games). 

Game  theory  is  a  branch  of  mathematics  which  studies  situations  of  conflict. 
Two-person  constant-sum  game  theory  models  are  useful  for  some  simple  militcoy 
problems;  but  satisfactory  theories  for  non-constant -sum  and  multi-person  games 
have  not  been  developed,  and  the  whole  theory  is  in  its  infancy.  Moreover,  the 
difficulty  of  calculating  solutions  to  game  theory  problems  severely  restricts 
the  number  of  variables  which  can  be  Included.  Consequently  there  is  a  strong 
tendency  to  substitute  games  for  game  theory,  and  games  have  been  developed 
which,  unlike  traditional  military  var  games,  permit  man>-  plays,  so  that  a 
number  of  possible  strategies  csun  be  tested  against  enemy  counter  strategies. 
Games,  like  game  theory,  are  not  completely  satisfactory,  but  for  other  reasons. 
Different  players  pla/  dlf ferently--BOffle  probably  too  well.  Sensitivity  analysis 
is  usually  impossible.  Results  are  therefore  very  difficult  to  Interpret. 

5.  Explicit  Treatment  of  Time  Phasing 

In  many  military  decisions  the  sequence  of  events  Is  of  critical  importance. 
For  example,  should  we  go  into  production  now  on  some  particular  missile  defense 
or  wait  two  years  until  a  better  one  is  developed? 

To  handle  such  problems  we  need  "dynaunlc"  models,  l.e.,  models  in  which 
the  variables  bear  dates.  We  have  such  models,  but  introducing  time  explicitly 
is  neither  easy  nor  painless: 

It  complicates  the  computation  by  multiplying  the 
number  of  variables.  If  we  put  time  in  we  have  t.  take 


something  else  out. 
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It  complicates  the  selection  of  a  criterion.  Solution 
A  may  be  better  for  '58,  worse  for  '6O;  Solution  B,  vice  versa. 

It  raises  in  acute  form  the  question  of  our  ability  to 
predict^  e.g.,  will  a  much  better  missile  really  be  ready  only 
two  years  later? 

6.  Broadening  of  Criteria 

The  selection  of  a  criterion  or  criteria  is  frequently  the  central  problem 
of  the  design  of  einy  systems  analysis. 

We  have,  characteristically,  numerous  alternative  ways  or  "systems"  for 
achieving  our  objectives  (positive  values).  All  involve  coats  of  some  kind 
(negative  values).  If  we  had  some  common  measure  for  all  the  positive  and 
negative  values  (as  a  business  firm  does,  more  or  less),  the  answer  to  the 
criterion  problem  would  be  obvious.  You  would  choose  the  system  which 
yielded  the  greatest  excess  of  positive  values  over  negative  ones. 

Unfortunately,  we  can't  do  this.  Objectives  and  costs  are  usually 
Incommensurate.  Objectives  are  likely  to  be  such  things  as  enemy  targets 
destroyed,  enemy  planes  shot  down,  probability  that  war  will  be  deterred.  Costs 
may  be  cirews  lost,  aircraft  lost,  millions  of  budget  dollars  expended.  I  don't 
want  to  exaggerate.  Frequently  we  can  find  common  denominators,  especially  on 
the  cost  side.  But  also  frequently,  especially  on  the  objectives  <3lde,  things 
just  won't  add. 

We  have  seen  that  In  the  typical  World  War  II  operations  analysis  problem 
fairly  simple,  obvious  ways  out  of  these  difficulties  could  usually  be  found. 

So  many  things  are  fixed  or  given.  Thus  In  the  bomber  formation  example,  we 
can  choose  the  formation  which  maximizes  target  destruction  for  given  aircraft 
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losaee,  or  minimizes  aircraft  losses  for  (£lven  tarejet  destruction.  These 
criteria,  although  they  sound  different,  are  logically  equivalent,  and  so  give 
the  same  answer  (for  the  same  scale  of  attack). 

Consider,  at  the  other  extreme,  the  mixed  force  composition  and  development 
problem  of  determining  the  optimum  composition  of  the  Air  Force,  I956  to  I966. 
What  are  Air  Force  objectives?  It  does  little  good  to  say  that  the  Air  Force 
objective  is  to  promote  such  national  objectives  as  to  win  a  war  if  there  is 
one,  or  to  deter  war.  What  kind  of  war?  When?  At  what  cost  in  lives  and 
dollars?  The  optimal  Air  Force  for  fighting  a  thermonuclear  war  is  not  optimal 
for  fighting  peripheral  wars.  Concentration  on  missiles  may  be  Just  right  for 
our  1966  capability,  but  may  weaken  us  for  i960.  While  our  criteria  must  be 
consistent  with  national  objectives,  they  must  be  defined  much  more  concretely, 
or  "operationally,"  No  one  has  written  (or  will  write)  a  real  long-range  war 
plane  to  guide  us. 

Of  course,  the  criterion  problem  In  the  typical  systems  analysis  is  not 
as  hard  as  In  this  case,  which  is  extreme.  It  is  extreme  because  a  decision 
on  force  composition  involves  optimizing  for  the  Air  Force  as  a  whole.  It  is 
a  decision  at  the  highest  Air  Force  level.  Decisions  at  this  level  are  almost 
certain  to  require  criteria  based  on  complex  and  uncertain  political  and 
strategic  factors.  But  many  Air  Force  decisions,  fortunately,  can  be  made  at 
lower  levelB--even  force  composition  and  development  decisions.  We  can  "factor 
out"  a  problem  and  the  variables  which  are  of  first-order  importance  in  its 
solution,  ignoring  the  rest.  And  instead  of  optimizing  for  the  Air  Force  as  a 
whole,  we  "sub -optimize"  for  some  sub-element  of  it,  choosing  an  appropriate 
lower  level  criterion.* 

♦For  a  fuller  discussion  of  this  point  see  my  earlier  article,  "Sub- 
Optimization  in  Operations  Pr jblems,"  Journal  of  the  Operations  Researcn 
Society  of  America,  Vol.  I,  No.  5,  May  1955.  P* 
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Dlatlasulahlng  problraa  which  w«  can  aucccaafulljr  factor  out  la  an  art. 

Lat  ac  lUuatrata  with  an  axai^Ia  froa  SAC  oparatlona.  The  choice  between  ZZ 
and  overaeaa  baaea  (or  what  coablnatlon  of  both)  won't  factor.  Overaeaa  baaea 
hare  treaendoua  raalflcationa  throushout  the  Air  Force  and  above  and  outalda 
It.  They  have  atrateglc  and  political  valxiea  and  vulnerabllltlea  at  other  and 
higher  levala  than  SAC  operatlona  that  we  would  be  atupld  to  Ignore  In  aakiog 
a  declalon.  Wa  can't  aub-optlalsa,  unleaa  we  frankly  recognise  that  the 
aub-optialsatlon  deala  only  with  mamm  of  the  factora  Important  to  the  declalon. 

On  the  other  hand^  given  that  we  want  at  leaat  part  of  SAC  baaed  In  the 

a 

ZI  (and  not  even  dependent  on  overaeaa  ataglng)>  we  think  we  can  auceeaafully 
factor  out  the  problem:  ahould  range  be  extended  by  bigger,  longer-ranga 
bombera,  or  by  air  refueling,  or  by  what  compromlae  between  the  two?  Thin 
choice  aeama  to  have  no  flrst>order  reparcuaaiona  on  air  dafenae,  on  any 
theater  operatlona,  on  International  polltlea,  etc.  It  can  be  aub-optlmlted  by 
chooalng  aa  an  appropriate  lover  level  criterion  the  moat  efficient  (in  tezma 
of  target  deatructlon)  uae  of  the  budget  or  reaourcea  made  available  for 
atrateglc  air. 

But  at  beat  criterion  aelectlon  la  bard--harder  In  force  compoaltlon  than 
In  operatlona  problena,  hardeat  In  developawnt  probleaa.  We  have  vreatlcd  with 
criterion  aelectlon  at  RAID  for  eight  yeara--vlth,  at  beat,  moderate  nucceaa. 

Ve  have  found  negative  rulea— criteria  to  avoid — but  few  poaltlve  onea  of 

i 

general  helpfulneaa.  There  la  no  aubatltute  for  good  Judgment,  and  no  aubatl- 
tute  for  exerelalng  it.  Working  out  a  ayatema  analyala  with  a  bad  criterion 
la  equivalent  to  anawerlng  tha  wrong  queatlon.  It  la  very  eaay  to  chooaa  a 
criterion  for  a  force  coq>oaltlon  or  developaMnt  problem  that  will  inaure  our 
having  the  optimal  ayatem  for  the  wrong  war  at  the  wrong  time  (to  be  fair,  it 
la  eaay  to  make  the  aama  mlatake  without  a  ayatema  analyala). 
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In  Bome  cases.  Indeed  frequently,  theire  is  no  sintjle  "right"  answer  to  the 
criterion  problem.  The  ultimate  values  of  decision-makers  differ  as  well  as  their 
image  of  the  future.  In  these  cases  the  systems  analyst  simply  has  to  conclude: 

If  you  want  an  Air  Force  which  will  do  A,  choose  Systems  X;  if  B,  choose  System  I. 

THK  DIFFICULTIES  ARE  M  THE  PROBLEMS 

It  is  clear  that  despite  our  boldness  and  ingenuity  in  developing  new 
techniques  of  analysis  since  the  war,  military  systems  analysis  as  it  exists 
today  is  an  Imperfect  instrument,  an  advisory  art  with  many  limitations. 

Because  we  have  so  boldly  tackled  long-range,  broad  context  problems 
of  force  composition  and  development,  we  have  made  drastic  "factoring"  and 
aggregation  Inevitable --with  no  good  rules  for  either.  We  have  learned  to 
make  extensive  calculations  of  Uncertain  contingencies,  but  still  don't  know 
what  to  recommend  (except  as  "senalble  men")  when,  as  is  almost  invariably  the 
case,  no  dominant  strategy  emerges.  We  have  developed  some  devices  for  dealing 
with  enemy  reactions  sind  problems  reguirlog  time  phasing,  but  only  at  the  high 
cost  of  further  factoring  or  aggregation  and  an  api)arent  further  complication  of 
the  already  intractable  criterion  problem.  In  the  matter  of  criteria  we  have  to 
be  satisfied  with  something  practical  which  falls  far  short  of  perfection. 

One  may  well  ask,  in  view  of  this  long  catalogue  of  difficulties,  dangers, 
and  limitations,  and  the  rather  obvious  possibilities  of  abuse  they  open  up, 
whether  military  systems  analysis  is  worth  supporting  and  continuing. 

The  first  thing  to  stress  in  answering  this  question  is  that  almost  all 
the  difficulties  we  have  discussed  are  Inherent  in  the  nature  of  the  military 
problems  that  systems  analysis  is  designed  to  help  solve,  and  by  no  means 
peculiar  to  systems  analysis,  however  we  define  it.  This  is  obviously  tme  of 
the  "many  variables"  difficulty.  In  most  of  the  military  problems  with  which 
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ve  are  concerned  lots  of  things  Just  are  Important.  Systems  analysis  has 
trouble  Including  all  of  them.  But  so  does  any  other  conceivable  approach. 

Systems  analysis  Is  Increasing  its  capability  to  handle  larger  problems  every 
year.  With  present  computers,  for  example,  ue  can  deal  with  far  more  intricate 
models  than  a  genius  can  manage  intuitively. 

Uncertainties  make  life  difficult  for  the  systems  analyst,  but  this  is  so 
because  the  problem  of  intelligent  behavior  under  uncertainty  is  really  hard. 

Systems  analysis  at  least  permits  one  to  explore  systematically  the  possibilities 
of  dominance  and  the  cost  of  Insurance. 

Enemy  reactions  are  hard  for  a  systeaiB  analysis  to  incorporate.  Game 
theory  is  in  its  Infancy.  War  gaming  in  any  form  has  shortcomings.  But  why? 
Because  the  problem  itself  is  so  hard.  The  enemy  probably  doesn't  know  himself 
how  he  will  act  or  react  in  I960.  How  can  we  outguess  him?  Hot  by  abandoning 
game  theory  and  war  gaming,  which  for  all  their  limitations  carry  us  further 
along  this  road  than  any  devices  yet  conceived. 

Time  phasing  Is  so  hard  that  few  systems  analyses  attempt  it  explicitly. 

Why?  Because  of  two  difficulties:  the  choice  of  criterion  when  payoffs  and 
costs  occur  in  different  years,  and  the  prediction  month  by  month  and  year  by 
year  of  changes  in  technology  and  other  parameters.  Both  are  intrinsic.  You 
can  get  rid  of  them  only  by  escaping  from  reality.  If  the  sequence  of  developments, 
or  their  speed.  Is  crucial  to  your  problem,  you  need  an  analysis  with  dates 
attached  to  every  variable  to  find  the  right  answer. 

But  noir  what  a  dynamic  systems  analysie  can  do  even  in  the  case  where  we 
can't  predict  the  speed  of  development  with  accuracy.  It  can  tell  us  what  the 
critical  development  speeds  are--lf  >  X,  wait;  If  <  X,  go  ahead — and  thus  enables 
us  to  focus  the  intuition  of  experts  on  a  manageable  technical  problem.  It  can 
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even,  in  Bone  cases,  yield  a  surprisingly  unequivocal  answer  despite  rather 
gross  uncertainties  about  speed  of  development.  We  have  encountered  such  cases 
at  RAND,  in  which,  while  postponement  might  have  resulted  in  startling  improvement 
in  sOme  performance  characteristics,  these  were  shown  by  systems  analysis  to  have 
trivial  military  worth. 

And  finally,  there  are  all  those  troublesome  problems  of  criteria.  I  can 
only  assure  you,  or  remind  you,  that  they  are  equally  troublesome  in  policy 
discussions  on  the  Air  Council,  in  the  Joint  Chiefs  of  Staff,  in  the  National 
Security  Council,  and  in  Congress.  You  don't  escape  from  them  by  escaping  from 
systems  analysis.  You  may  be  able  to  fuzz  them  up  or  conceal  conflicts  In  a 
clever  debate  or  essay  (you  can  do  the  same  in  a  clever  systems  analysis),  but 
they  remain  for  a  clever  opponent  to  reveal.  The  fundamental  difficulty  is  that 
there  does  not  exist  a  clear  cut,  definitive,  operationally  meaningful  statement 
of  national  objectives  or  of  Air  Force  obJectlve8--even  for  the  present,  let 
alone  for  1960-1965* 


THE  POSITIVE  SIDS 

Before  we  can  say  anything  in  general  about  the  usefulness  of  systems 
analysis  we  must  know  what  we  are  contrasting  it  with.  If  we  define  systems 
analysis  broadly  to  Include  the  various  game  techniques,  etc.,  discussed  In 
the  preceding  secltons,  what  are  the  alternatives? 

Let's  consider  two.  Concentrating  on  the  first  word  "systems,"  the 
alternative  is  unsystematic  or  piecemeal  consideration  of  problems. 

Concentrating  on  the  second  word  "analysis,"  the  alternative  is,  I 


suppose.  Intuition 
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1.  Systems  Versus  Nonsystems 

This  distinction  has  nothing  to  do,  necessarily,  with  analysis.  It  is  a 
question  of  breadth  of  context.  In  principle,  one  can  attempt  to  intuit  answers 
in  a  broad  or  narrow  context  or  use  analysis  in  a  broad  or  narrow  context. 

It  would  be  foolish  to  maintain  that  broad  systems  contexts  are  good, 
narrow  contexts  bad.  It  all  depends  on  the  problem.  Is  it  factorable  or  not? 
How  factorable  la  it?  Systems  contexts  can  be  too  broad,  and  when  they  are, 
they  are  wasteful.  You  pay  a  hea-vy  price  for  a  broad  context.  For  anything 
you  put  in  an  analysis,  something  must  come  out.  The  broader  the  context,  the 
less  detail.  If  you  are  a  scientist  trying  to  develop  materials  to  withstand 
the  heat  of  rocket  engines,  your  chances  of  success  will  be  reduced  to  the 
extent  that  you  devote  time  and  energy  to  pondering  the  relative  likelihood 
of  big  and  little  ware.  As  a  matter  of  historical  fact  almost  all  scientific 
and  technological  progress  has  been  achieved  within  very  narrow  contextB--by 
scientists  wearing  blinders.  Let's  continue  a  fruitful  division  of  labor  and 
not  all  become  systems  analysts.  Some  of  us  are  a  little  concerned  that  a 
Large  proportion  of  our  best  design  engineers  in  aircraft  companies  seem  to 
be  spending  their  time  designing  systems  analyses  instead  of  aircraft. 

Nevertheless,  there  are  cases  where  the  systems  approach--the  systematic 
examination  of  broad  alternatlve8--throws  a  flood  of  light  on  important  problems. 

Our  previous  example  is  such  a  case.  What  methods  of  range  extension 
should  be  used  to  enable  U.S.  bombers  to  reach  targets  deep  in  Russia?  The 
broad  alternatives  are:  overseas  operating  bases  with  medium  bombers; 
overseas  bases  for  staging  only;  big,  very  long-range  bombers;  mother-daughter 
arrangements;  air  refueling,  etc.  Because  the  alternatives  are  broad,  they 


need  to  be  examined  in  a  broad  context.  When  we  do  we  discover  that  some 
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systems  have  a  superiority  of  two  to  one  to  five  to  one  aver  other  plausible 
systems  with  enthusiastic  advocates.  This  is  a  tremendously  important 
conclusion.  It  could  not  have  been  reached,  or  at  least  not  demonstrated, 
without  a  comprehensive  systems  approach  and  systems  costing. 

Let  me  give  you  another  example  of  a  different  sort  that  cropped  up  In 

a  RAHD  defense  study  of  several  years  ago.  It  was  at  that  time  operational 

a 

doctrine  for  certain  intere^tors  to  carry  armament  that,  according  to  Air 
Defense  Command  estimates,  gave  each  plane  a  fifty  per  cent  probability  of 
killing  an  intercepted  bomber.  Well,  fifty  per  cent  looked  mighty  good  to 
most  experienced  Air  Force  hands.  It  was  lots  better  than  euvythlng  achieved 
in  World  War  II. 

What  did  we  find  when  we  examined  this  doctrine  in  a  systems  context? 
Esaentlally; 

(a)  As  was  not  the  case  In  World  War  II,  we  were  really  preparing 
for  defense  against  one  (or  at  most  a  very  few)  massive  atomic 

strikes. 

(b)  The  total  systems  cost  of  procuring  and  operating  the  Inter- 
ceptora--to  get  them  Into  position  prepared  to  fire  a  rocket 

at  an  incoming  boraber--wa8  extremely  high,  so  high  that  the  most 
lavish  expenditure  on  armament  scarcely  affected  the  total. 

(c)  It  was  therefore  obvious  nonsense  to  economize  on  armament. 

(d)  ^e  single  pass  kill  probability  and  the  kill  potential  of  the 
defense  system  could  be  Increased  by  nearly  flfy  per  cent 

simply  by  increasing  the  armament.  The  performance  degradation  of 
the  interceptor  resulting  from  increased  weight  was  of  the  order 
of  five  per  cent--wbich,  at  least  In  the  period  of  interest--had  a 
negligible  impact  on  kill  potential. 
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Now  this  again  waa  a  tremendoualy  Important  result  of  looking  at  a  problem 
In  a  systema  context.  A  ayatema  analysis  wasn't  really  necessary.  1  am  sure 
that  some  Air  Force  officers,  using  a  broad  aystema  context,  thought  their  way 
through  this  one  and  reached  the  right  conclusion  without  so  much  ae  using  the 
back  of  an  envelope.  But  many  apparently  didn't,  and  the  doctrine  wasn't 
changed  until  the  systems  analysis  waa  produced  and  presented.  Systems  analysis 
forces  both  the  systema  analyst  and  his  audience  to  think  the  problem  through 
In  a  systema  context. 

2.  Analyala  Versua  Intuition 

Let  us  turn  to  the  second  part  of  my  comparison;  analysis  versus 
Intuition. 

The  main  point  I  want  to  make  might  be  called  the  inevitability  of  analysis. 
What  we  call  Intuition  is  a  species  of  logical  analysis.  It  uses  models.  In  our 
sense  of  simplified  conceptual  counterparts  of  reality.  Not  surprisingly,  in 
military  problems  as  in  so  majiy  others,  it  Is  sometimes  useful  to  buttress  our 
feeble  minds  with  some  external  assistance:  a  pencil  and  the  back  of  an 
envelope;  a  few  equations;  a  desk  calculator;  sophisticated  statistical  and 
mathematical  theory;  high-speed  calculators. 

How  far  you  go  with  such  aids  depends  on  the  problem.  But  very  frequently 
they  enable  you  to  find  a  solution  you  couldn't  otherwise  find,  or  to  demonstrate 
that  your  intuitive  solution  was  wrong--or,  what  is  sometimes  as  important, 
right. 

I  am  not  selling  intuition  short.  The  unaided  human  mind  is  quite 
remarkably  proficient  at  solving  some  kinds  of  problems.  Let  me  remind  you: 

(a)  Some  human  beings  play  very  good  games  of  chess.  No  machine 
can  yet  give  them  a  match. 
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(b)  Human  beings  at  RAIID  with  Intuition,  pins,  and  a  piece  of  string 
found  the  optimal  route  in  the  Traveling  Salesman  problem  out  of 

10^^  possible  routes. 

(c)  On  the  evening  of  last  November  second  the  Columbia  Broadcasting 
System  tised  two  methods  for  predicting  the  election  results  from 

the  very  early  returns. 

(1)  The  intuition  of  assorted  political  experts. 

(2)  A  complex  multi-variable  model  calculated  on  a  UNIVAC. 

The  UNIVAC  was  grotesquely  wrong:  the  experts  did  not  do  too  badly. 

The  human  mind  has  some  great  advantages  over  any  machine--lf  we  think  of 
them  as  rivals  or  alternatives.  It  has,  by  comparison,  a  wonderfully  capacious 
memory,  which  enables  it  to  learn  from  experience.  It  has  a  marvelous  facility 
for  factoring  out  the  important  variables  and  suppressing  or  aggregating  the 
rest.  Closely  related  to  this  facility,  it  can  build  models  highly  appropriate 
to  the  particular  problem  it  is  considering.  Big  formal  models  computed  on 
machines  are  much  less  flexible.  These  are  the  reasons  human  beings  beat  machines 
at  chess  or  war  games. 

But,  on  the  positive  side; 

(a)  It  is  utterly  wrong  to  look  upon  intuition  and  analysis  or 
minds  and  machines  as  rivals  or  alternatives.  Properly  used, 

they  complement  each  other.  We  have  seen  that  every  systems  analysis 

is  shot  through  with  Intuition  and  Judgment.  We  have  experimented  at 

RAND  with  man-machine  combinations  which  will  play  war  games  better 

than  either  men  or  machines. 

(b)  While  unaided  intuition  Is  sometimes  strikingly  successful,  as 


It  was  last  November  second,  or  In  the  Traveling  Salesman 
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problem.  It  can  aleo  fall  flat  on  Ite  face.  For  example,  in  the 
election  of*  November,  1952,  when  the  UHIVAC  va*  dead  right,  and 
the  Intuition  of  the  expert*  so  wrong  that  they  euppress^  the 
UNIVAC  answer. 

In  contrast  to  the  Traveling  Salesman  problem,  try  your 
intuition  on  this:  there  are  25  persona  in  a  group.  VHiat  are 
the  chances  that  at  least  two  have  the  same  birthday?  Almost 
everyone  without  statistical  training  eays--very  small.  In  fact, 
they  are  better  than  even.  If  there  are  sixty  in  the  group,  the 
chances  are  99*^  per  cent! 

(c)  One  of  the  troubles  with  intuition  is  that  you  never  know 
whether  It  is  good  or  not  without  an  analytic  check.  For 

example,  our  intuition  was  good  enough  to  solve  the  Traveling 
Salesman  problem,  but  we  didn't  know  it  until  we  solved  it 
analytically  in  a  linear  programming  formulation. 

MATS  didn't  know  its  assignment  of  aircraft  to  routes  was 
within  five  per  cent  of  optimal  until  we  worked  out  a  linear 
programming  solution. 

I've  eaid  there  are  good  chess  players.  But  we  d  m't  really 
know.  Maybe  even  the  best  are  as  far  from  optimal  strategies  as 
expert  opinion  so  frequently  has  been  on  military  problems. 

(d)  Finally,  analytic  and  computing  techniques  enable  us  to  do 
things  we  otbei^lse  couldn't.  They  may  be  poor  on  the  memory 

side,  but  they  have  some  capabilities  unaided  human  minds  don't. 

Look  again  at  the  UNIVAC  flascoe  on  election  eves,  1952  and  195**. 
Here  we  had  an  elaborate  model  and  a  hlgb-apeen  computer.  In  1952 


P-699 

3-18-55 

-25- 

it  was  able  to  take  the  first  few  precinct  returns  from  eastern  states 
and  trace  their  con8equence8--on  its  built-in  assumption  that  similar 
trends  were  running  in  all  precincts  throughout  the  nation.  They  were, 
and  the  answer  was  dead  right.  Ttie  experts  couldn't  carry  out  such  a 
calculation  in  their  heads,  and  were  inclined,  like  most  experts,  to 
err  on  the  "conservative"  side. 

Or  look  at  a  different  kind  of  example--of  insights  derived 
from  theory.  Take  a  brand  new  theory-one  that  1  described  as  in 
its  infancy  and  of  very  limited  usefulness--vlz. ,  game  theory. 

In  connection  with  RAND  defense  studies  we  have  long  been 
Interested  in  the  optimal  deployment  of  limited  defenses  among 
targets,  some  of  which  are  more  valuable  than  others.  Unfortunately, 
we  have  found  no  satisfactory  general  rule  for  deploying  defenses, 
hut  game  theory  has  given  us  valuable  insights  and  hints  about  good 
£Lnd  bad  deployments  we  would  not  otheirwlse  have  had. 

One  striking  example:  Suppose  you  have  your  defenses  deployed 
as  well  as  you  can.  Now  you  get  more  defenses.  How  do  you  deploy 
them: 

Well,  my  intuition  told  me  (and  so  did  most  people's)  that  you 
deploy  them  mainly  to  protect  additional  target8--additional  cities, 
harbors,  airbases,  etc.,  tha.t  you  didn't  previously  have  enough  stuff 
to  defend. 

Game  theory  says  no.  You  use  additional  defenses  mainly  tj  increase 
the  defense  of  targets  already  defended.  In  fact  over  a  wide  range, 
the  more  you  have  the  more  you  concentrate  it. 
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Having  been  Informed  of  this  startling  result,  you  think  about 
it  and  begin  to  see  the  rationale.  An  increase  in  your  defensive 
strength  is  equivalent  to  a  decrease  in  the  enemy's  offensive  strength. 
But  as  his  strength  decreases,  he  has  to  concentrate  more  and  more  on 
your  most  valuable  targets  to  achieve  anything  worth  while.  These  are 
the  ones  on  which  you  therefore  have  to  concentrate  your  defense. 

But  intuition  alone  would  not  have  told  us  this.  At  any  rate, 
not  unequivocally  enough  to  lead  one  to  act  on  it. 


COHCLUSIOM 


Does  analysis  help  more  in  the  narrow  context  problems,  where  it  has 
commonly  been  applied  by  scientists,  or  in  the  broad  context  problems,  which 
are  the  special  province  of  systems  analysts? 

I-Juu'L  know.  On  the  basis  of  results,  certainly  one  would  have  to  say 
that  the  case  for  analysis  in  broad  context  problems  is  comparatively  unproved. 


1  ■ifa.me 


,  however, "^^^gest^ 


ne  reason  why,  when  we  are  dealing  with  broad  problems 


with  broad  systems  analyses,  explicit  analysis  using  explicit  models  can  be 

especially  important.  — - - —  ~  "■  - 

We  trust  a  man's  intuition  in  a  field  in  which  he  Is  expert?^ But  in  these 
cases  we  are  dealing  with  a  field  so  broad  that  no  one  can  be  called  expert.  A 
typical  systems  analysis  depends  critically  on  numerous  technological  factors  in 
several  fields  of  technology;  on  military  operations  and  logistics  factors  on 
both  our  Bide  and  the  enemy's;  on  broad  economic,  political  and  strategic  factors 
and  on  quite  Intricate  relations  among  all  these.  No  one  is  an  expert  in  more 
than  one  or  two  of  the  sub-fields;  no  one  is  an  expert  In  the  field  as  a  whole 


and  the  Interirelati ons.  So  no  one's  unsupported  Intuitions  in  such  a  field  can 


be  trusted. 


Systems  «aaly8es  should  be  locked  upon  not  as  the  antithesis  of  Judgment  but 
\  ' 

as  a  framework  which  permits  the  Judgment  of  experts  in  numerous  sub-fields  to 
be  combined-^to  yield  results  »/tilch  transcend  any  individual  Judgment.  This  is 
its  aim  ajid  opportunity.  ' 

•5 

But  we  still  have  the  qudlftltm:  Where  is  the  "expert"  in  the  field  as  a 
whole  with  the  Judgment  required  to  design  a  systems  analysis  and  interpret  its 
results?  We  know  there  are  not  any  real  experts.  But  we  think  we  can 
demonstrate  that  the  degree  of  expertness  required  to  design  a  systems  analysis 
is  less  than  the  degree  of  expertness  required  to  intuit  a  fcjod  answer  without 
a  systems  analysis. 

Let  me^  put  it  in  another  way.  We  tend  to  be  worse,  in  an  absolute  sense, 

In  applying  analysis  or  scientific  method  to  broad  context  problems;  but 
xmalded  intuition  in  such  problems  Is  also  much  worse  in  an  absolute  sense. 

Let's  not  deprive  ourselves  of  any  useful  tools,  however  short  of  perfection 
they  may  fall. 
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